Theory and Practice in Making Seamless Stereo Mosaics from Airborne Video

In this paper we present a novel method for automatically and efficiently generating stereoscopic mosaics by registration of optical data collected by a video camera mounted on an airborne platform that mainly undergoes translating motion. The resultant mosaics are seamless and will exhibit correct three-dimensional (3D) views. The basic idea is to fast construct stereo mosaics before 3D recovery for applications such as image-based rendering and environmental monitoring. Image mosaicing from a translating camera will result in a mosaic image with multiple viewpoints on a long motion path. It raises a set of different problems from that of circular projections of a rotating camera, including motion models, motion estimation, generation of geometric seamless image mosaic with large inter-frame motion parallax, and epipolar geometry of multiperspective projection. First we will analyze two possible geometric representations for stereo mosaics-multiple-perspective mosaics and parallel-perspective mosaics. We will show that the parallel-perspective mosaic representation is superior to a multiple-perspective representation for the geometric seamlessness of its visual appearance and the simplicity of its epipolar geometry, which will benefit both interactive human viewing and automated stereo matching. Then we will show that the condition under which such stereo mosaics can be created is a rather general motion(3D rotation and translation) with a dominant direction. Next, a novel method based on interframe local matches and parallel-perspective view synthesis is proposed to generate dense and seamless parallel-perspective stereo mosaics in the presence of motion parallax. The advantages of stereo mosaics in representation, computation, 3D recovery resolution and 3D visualization are discussed throughout the paper. Experimental results on long video sequences will be given to validate our analysis.